from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.073	0.059	0.0	0.017	-1	1	0.663	3.407	0.042	5.012	0.065	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.073	0.059	0.0	0.017	-1	1	0.663	0.025	0.002	679.466	45.496	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.320	0.007	0.0	0.320	-1	1	1.000	3.407	0.042	0.094	0.002	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.320	0.007	0.0	0.320	-1	1	1.000	0.025	0.002	12.738	0.898	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.356	0.025	0.0	0.017	-1	5	0.757	4.238	0.114	4.095	0.110	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.356	0.025	0.0	0.017	-1	5	0.757	0.027	0.002	642.185	54.702	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.328	0.007	0.0	0.328	-1	5	1.000	4.238	0.114	0.077	0.003	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.328	0.007	0.0	0.328	-1	5	1.000	0.027	0.002	12.122	1.069	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.457	0.015	0.0	0.017	1	100	0.882	2.146	0.028	8.137	0.105	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.457	0.015	0.0	0.017	1	100	0.882	0.022	0.000	794.931	16.040	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.322	0.006	0.0	0.322	1	100	1.000	2.146	0.028	0.150	0.003	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.322	0.006	0.0	0.322	1	100	1.000	0.022	0.000	14.650	0.397	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.184	0.020	0.0	0.017	-1	100	0.882	4.201	0.054	4.091	0.053	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.184	0.020	0.0	0.017	-1	100	0.882	0.025	0.002	683.318	52.633	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.325	0.010	0.0	0.325	-1	100	1.000	4.201	0.054	0.077	0.003	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.325	0.010	0.0	0.325	-1	100	1.000	0.025	0.002	12.914	1.072	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.190	0.106	0.0	0.017	1	5	0.757	2.105	0.015	8.168	0.076	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.190	0.106	0.0	0.017	1	5	0.757	0.022	0.002	767.303	83.013	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.321	0.011	0.0	0.321	1	5	1.000	2.105	0.015	0.153	0.005	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.321	0.011	0.0	0.321	1	5	1.000	0.022	0.002	14.332	1.621	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.320	0.082	0.0	0.017	1	1	0.663	1.337	0.016	12.958	0.164	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.320	0.082	0.0	0.017	1	1	0.663	0.024	0.002	734.245	66.570	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.323	0.010	0.0	0.323	1	1	1.000	1.337	0.016	0.242	0.008	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.323	0.010	0.0	0.323	1	1	1.000	0.024	0.002	13.699	1.316	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.060	0.035	0.0	0.004	-1	1	0.896	3.152	0.040	1.288	0.020	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.060	0.035	0.0	0.004	-1	1	0.896	0.008	0.006	511.869	393.592	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.261	0.006	0.0	0.261	-1	1	1.000	3.152	0.040	0.083	0.002	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.261	0.006	0.0	0.261	-1	1	1.000	0.008	0.006	32.941	25.338	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.141	0.025	0.0	0.004	-1	5	0.922	3.935	0.035	1.052	0.011	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.141	0.025	0.0	0.004	-1	5	0.922	0.010	0.006	399.905	215.208	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.259	0.005	0.0	0.259	-1	5	1.000	3.935	0.035	0.066	0.001	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.259	0.005	0.0	0.259	-1	5	1.000	0.010	0.006	25.056	13.492	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.135	0.062	0.0	0.004	1	100	0.929	1.928	0.020	2.144	0.039	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.135	0.062	0.0	0.004	1	100	0.929	0.003	0.000	1299.720	110.141	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.262	0.007	0.0	0.262	1	100	1.000	1.928	0.020	0.136	0.004	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.262	0.007	0.0	0.262	1	100	1.000	0.003	0.000	82.286	7.242	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.088	0.037	0.0	0.004	-1	100	0.929	3.923	0.029	1.042	0.012	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.088	0.037	0.0	0.004	-1	100	0.929	0.010	0.005	408.543	187.508	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.265	0.008	0.0	0.265	-1	100	1.000	3.923	0.029	0.067	0.002	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.265	0.008	0.0	0.265	-1	100	1.000	0.010	0.005	26.438	12.157	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.057	0.075	0.0	0.004	1	5	0.922	1.904	0.011	2.130	0.041	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.057	0.075	0.0	0.004	1	5	0.922	0.003	0.000	1333.320	104.502	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.265	0.007	0.0	0.265	1	5	1.000	1.904	0.011	0.139	0.004	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.265	0.007	0.0	0.265	1	5	1.000	0.003	0.000	87.068	6.986	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.040	0.043	0.0	0.004	1	1	0.896	1.109	0.012	3.643	0.055	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.040	0.043	0.0	0.004	1	1	0.896	0.002	0.000	2043.365	126.836	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.261	0.006	0.0	0.261	1	1	1.000	1.109	0.012	0.235	0.006	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.261	0.006	0.0	0.261	1	1	1.000	0.002	0.000	132.034	8.623	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	141.039	0.000	0.0	0.141	-1	1	0.929	0.522	0.013	270.273	6.796	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	141.039	0.000	0.0	0.141	-1	1	0.929	0.756	2.258	186.665	557.841	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.876	0.218	0.0	2.876	-1	1	1.000	0.522	0.013	5.511	0.441	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.876	0.218	0.0	2.876	-1	1	1.000	0.756	2.258	3.806	11.377	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	139.338	0.000	0.0	0.139	-1	5	0.946	0.974	0.012	143.128	1.723	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	139.338	0.000	0.0	0.139	-1	5	0.946	0.004	0.001	38572.494	10159.877	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.830	0.112	0.0	2.830	-1	5	1.000	0.974	0.012	2.907	0.120	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.830	0.112	0.0	2.830	-1	5	1.000	0.004	0.001	783.460	208.674	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	153.506	0.000	0.0	0.154	1	100	0.951	5.651	0.024	27.165	0.116	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	153.506	0.000	0.0	0.154	1	100	0.951	0.004	0.001	37308.483	11740.279	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.851	0.153	0.0	2.851	1	100	1.000	5.651	0.024	0.505	0.027	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.851	0.153	0.0	2.851	1	100	1.000	0.004	0.001	693.021	221.242	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	158.211	0.000	0.0	0.158	-1	100	0.951	3.276	0.040	48.300	0.585	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	158.211	0.000	0.0	0.158	-1	100	0.951	0.006	0.001	26926.305	5460.622	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.820	0.077	0.0	2.820	-1	100	1.000	3.276	0.040	0.861	0.026	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.820	0.077	0.0	2.820	-1	100	1.000	0.006	0.001	479.985	98.207	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	155.224	0.000	0.0	0.155	1	5	0.946	1.676	0.013	92.627	0.712	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	155.224	0.000	0.0	0.155	1	5	0.946	0.002	0.000	88894.350	23242.936	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.842	0.066	0.0	2.842	1	5	1.000	1.676	0.013	1.696	0.041	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.842	0.066	0.0	2.842	1	5	1.000	0.002	0.000	1627.322	427.156	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	158.078	0.000	0.0	0.158	1	1	0.929	0.884	0.016	178.806	3.320	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	158.078	0.000	0.0	0.158	1	1	0.929	0.001	0.000	118503.695	34384.285	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.849	0.105	0.0	2.849	1	1	1.000	0.884	0.016	3.222	0.133	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.849	0.105	0.0	2.849	1	1	1.000	0.001	0.000	2135.668	624.642	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.042	0.003	0.0	0.000	-1	1	0.891	0.028	0.001	1.486	0.137	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.042	0.003	0.0	0.000	-1	1	0.891	0.003	0.000	14.887	2.667	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.028	0.001	0.194	0.012	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.003	0.000	1.945	0.323	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.001	0.0	0.000	-1	5	0.911	0.032	0.004	1.297	0.178	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.001	0.0	0.000	-1	5	0.911	0.003	0.001	14.956	4.475	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	5	1.000	0.032	0.004	0.181	0.027	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	5	1.000	0.003	0.001	2.088	0.636	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.063	0.003	0.0	0.000	1	100	0.894	0.039	0.001	1.616	0.083	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.063	0.003	0.0	0.000	1	100	0.894	0.001	0.000	75.986	17.949	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	100	1.000	0.039	0.001	0.143	0.009	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	100	1.000	0.001	0.000	6.725	1.605	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.062	0.003	0.0	0.000	-1	100	0.894	0.042	0.004	1.458	0.163	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.062	0.003	0.0	0.000	-1	100	0.894	0.003	0.001	22.721	5.623	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	-1	100	1.000	0.042	0.004	0.144	0.037	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.001	0.0	0.006	-1	100	1.000	0.003	0.001	2.239	0.764	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.001	0.0	0.000	1	5	0.911	0.027	0.001	1.507	0.073	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.001	0.0	0.000	1	5	0.911	0.001	0.000	45.153	7.707	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	5	1.000	0.027	0.001	0.211	0.016	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	5	1.000	0.001	0.000	6.308	1.144	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.003	0.0	0.000	1	1	0.891	0.029	0.012	1.431	0.591	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.003	0.0	0.000	1	1	0.891	0.001	0.000	51.261	9.594	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	1	1.000	0.029	0.012	0.194	0.080	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	1	1.000	0.001	0.000	6.943	1.323	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	n_iter	mean_duration	std_duration	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.466	0.017	0.002	0.000	0.824	0.155	0.005	3.014	0.144	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.466	0.017	0.002	0.000	0.824	0.020	0.002	23.379	2.308	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.383	0.012	0.000	0.383	1.000	0.155	0.005	2.478	0.110	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.383	0.012	0.000	0.383	1.000	0.020	0.002	19.217	1.867	See	See